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Operations — Give me all your datz 


Operations is increasingly about End-to-End control across multiple 
networks/services; across multiple layers in network, computing, and 
software stacks; and, across a variety of time-frames. It is, therefore, a 
problem of integration of huge amounts of very heterogeneous data in 


. real time. 
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“Half Life” and “Inconvenience Threshold” 


"Kalf-Life' of "Inconvenlance Threshold" 
Parceived Parsonal Travel Distance to Obtaln 
Information Value Valuable Information 
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Each communication technology advance has shortened the useful life of information and —__ 
increased the need to obtain new information more rapidly regardiess of the situation or location... 
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Operations Today 


Operations 


ai ATT 9G 6:44 PM as 


Applications speakdit 


MOBILE LOCAL SEARCH 





= 





Network 








~—ismwaos ] | eas ne on 
\ —* | a 
















7 CMBRMAO1 





*) NYCMNY54 


















Ny — 
nk 
sy 
x 
\ —FT 
LSANCA03 *\ 





. PHNXAZMA | 
SNDGCA02 









Page 4 4/27/2010 2010 Copyright AT&T Corporation All Rights Reserved 


—_—_— ese 
Operations Cycle 


Each phase can be complex in 
large networked systems 


° Monitoring involves data across 
multiple hosts and multiple 
SOUICEeS. 

e — Analyzing may involve heuristics or 
evaluation over time. 


} Monitor 





. Decision may involve evaluating 
Control Analyze tradeofts or distributed algorithms. 


¢ — Control may involve distributed 
coordination across multiple hosts. 





Decide — 
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Operations Cycle 


This often involves huge 
amounts, and many types, 
of data: 


¢ Examples may include: 


e Circuit Switched Networks 
— SS7, CDR, ... 

e IP Networks - Netflow, 

Control Analyze SNMP, ... 

° Mobile Networks — UE, 
Physical & Location, 
RNC/MSC, ... 

e Systems — Logs, 
utilization, virtualization, 


» Monitor 








Decide — 


‘N e Sensors — SmartGrid, 
TeleHealth, RFID, ... 
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Systems Environment 
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Network Based Computing: 


Goal - Adaptive Maintenance 
MTTR 
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scale includes volume, volatility, complexity, reliability, and 
security. 
3,600,000,000,000,000,000,000 





Report on American Consumers 
Roger E. Bohn & James E. Short 
Global Information Industry Center, UCSD 


Industry—University Collaborations 
AT&T, Cisco Systems, IBM, Intel Corporations, 
LSI, Oracle, Seagate Technology 
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How Much Information? 2009 


e The goal of HMI? Project is to create a census 
of the world’s data and information 


e First Report: Information at the US Consumer 
Level 


e How Much Information was Consumed by 
Individuals tn the U.S. in 2008? 


3.6 Zettabytes (ZB = 10%21 bytes) 


eHow much ts 3.6 Zettabytes? If we 
printed 3.6 zettabytes of text in books, and 
stacked them as tightly as possible across the 


United States including Alaska, the pile would 
be 7 feet high 
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How Much Information? 2009 


e Measures of information include all data delivered to 
people, whether for personal consumption, for 
communication or for any other reason: Cable TV, 
Broadcast TV, Radio, Telephone Line, Internet, 
Wireless, etc. 


Network Data 


e The AT&T Network 


e Currently carries 18.7 Petabytes of data traffic on an 
average business day (PB = 1015 bytes) 


e Nearly 5 Billion calls per day 
Information in Flight 
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Existing delivered information is estimated to be O(Zettabytes) 
per year. Most of the new information, and much of the existing 
is, IN any year, on a network — 1.e. in flight. Video is responsible 
for the enormous growth in total amount of information in flight. 
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Data: Movement, Management, Ananmpe@s 
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The scale and power of data related technology continues to 
increase dramatically. Volume and complexity that would 
have seemed impossible just a few years ago Is now 
routine. The next bottleneck is Information. 
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Analysis/Visualization on a Large Graph 
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AT&T InfoLab 


NETWORK 









COMPUTER 


SCIENCE Fraud, Customer Focused Operations, 
Mobility Network, Hosting, 
Sensor Networks , Security, Marketing, ... 









Transport 


That operate at scale, 
and in real time DATA 
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Can we completely automate the staging of 
data for analysis (From origination to analysis 
ready)? 








Collection 
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Transformation Storage & Mining & 
& DB Analysis 
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Example: Can the IP Network Run Itself? 





e How to push automation as far as possible? 


e Smart, transactional network update 
e Deep Just in Time Parse, Semantic Extraction of Router Configurations 
e Integrated with Ticketing, Network Databases, Monitoring, Overarching 


Timely, accurate information Is essential!!! 


Tools that separate capabilities from policies, since policies can change 


fast 


e Example: link utilizations should be < 80% except for links involved in that new 
VoIP trial in Phoenix with vendor X equipment, where utilizations should be < 


50%, except for ... 


Big Guard Rails — extensive monitoring and info correlat'e7 /elidetian 
Huge Operations involvement at every step 


e Simpler, repeatable tasks/repairs automated 


Policy Control 


e State Transition Paradigm (Policy is Stateful!) 
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Providing Traffic Analysis for Large IP Networks 


Robust storage and 
presentation tools 


provide on-demand _ . : AT&T 
reporting via web ———— | 7 
interface 







100% of entiré network, 7X24, 
months of data online 


Powerful Smart 
Sampling on TAS 
collectors reduces data 
volumes 
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This Takes You From... TO... 


e From networks consisting of numerous, uncoordinated, error-prone 
systems 


e e.g., Manual grappling with the changing state of the art in packet 
filters, route maps, IP, MPLS routing, layer 1-3 interworkina, ... 


e To networks where operators leverage automated network-wide 
views to assure performance 


e e.g., assure negligible customer impact from planned cable intrusion 
scheduled tonight in New Mexico at midnight, mountain time” 


e To networks where designers leverage automated mechanisms for 
real-time network response 


°e e.g., Survive any single fiber or router failure with all link utilizations < 
70%" 


e To networks where VPN customers can leverage detailed views of 
their traffic and flexible, policy-driven routing 


e e.g., ‘traffic from east coast customer X sites should prefer X’s Dallas 
data center and X’s Phoenix Internet Gateway” 
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Large Scale Data Management 
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DSMS + DBMS 


e DBMS - Data Base Management System 
e Query data stored on disk. 
e Load via transactions or batch update. 
e Queries can be complex 
e Expensive joins, historical data, etc. 
e DSMS - Data Stream Management System 
e Analyze records “in flight” — very cheap data ingest. 
e Relatively simple queries 
e Must execute in real time. 
e Limited historical analysis. 
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Network Monitoring via GSQU LOEEuw 


e Simple 
Select tb, SrcIP, DestIP, count (*) 
From TCP 


Group By tb, SrcIP, DestIP 
e Flexible 


e Auto-configuration to monitor selected network interfaces. 
e Complex queries via composition. 


e Optimizable 


“hh 
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Data, data, everywhere! 


Incredible amounts of data Vast amounts of chaotic ad 
are stored in well-behaved hoc data from logs, legacy 
formats: systems, etc. exist as well. 
Databases: 


Database 


XML: 








Data Transformation: 
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Ad hoc Data 


PADS Data 
Description 


Standard formats & schema; 
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Examples of Ad Hoc Data 


It is difficult to define ad hoc data precisely: it is less formalized than data in relational 
databases or xml, but more structured than free-form text. To give a feel for the 
variety, we give some examples. 


Binary data: DNS packets 


QOQQQ0000: Oro 
QOQQQ0O010: 6d00 
VOODOUZU: 00277 


QOQQ00030: 7465 
OQQ00040: 8400 





ASCII data: Option price trading authority 


HLOOQOQQOQOQOO2ZSTART OF OPEN INTEREST 

d QOOQOOQOQQO03FZYX GLABOOOOO 30000300000 

HMOOQOQQQQO4END OF OPEN INTEREST 

HEOQQQOQOQO5START OF SUMMARY 

F QOQQQQQQONYZX BLIOBOONNSZ0001Z00000 7O000BO0005S0Q0000000520000 
OO490Q00005100+00000L00BOQQ00005300000052500000535000 

HFOOQOQOQQOQQ7END OF SUMMARY 


k QQOQOQQQOQO8LYXW BLKBOQQQD65SGOQOQNQV09IDOLONQQOQOLOQVDOZONONVO 








The amount and variety of data are far too great for all but 

the most powerful tools. Robust, automated analysis is 

essential to control in real time. Humans are still the best 

visual pattern recognition engines available. 
V0—_—uGeQeyueaeNRe_eSSSSSSSSSm9@@0@000SSml 


An example of a small system 
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Visualizer 
Managing Systems Distributed Worldwide 
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e Early identification of trends 


e Dramatic reduction of unnecessary alarms 


e Complete drill down for fast resolution of 


problems 


Control charts identify unusual conditions 


e Visual query capability 


e All views, e.g. physical/logical, related 
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Enhanced Control of Networks and Servers 


¥VizGEMS 6.0 - Microsoft Internet Explorer 
File Edit View Favorites Tools Help 
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Topview: exploring large networ 
topological zooming 


@@ GraphEmbedder Layout 


File Parameters Stressparameters Style Statistics Zoom Embedder Ejigenprojection Stress MDS 


Eigen embedder Focus Fisheye Nofocus Coarsen 


e Large networks are everywhere. 


e Visualization problem: complexity, clutter, 
occlusion, visual overload 








e Topview: interactive adjustable focus a 
- high level of detail near a focus 0-1. pp 


- progressively less detail further away 
the view always reflects the original 
structure “semantic zooming” 
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Talkalytics 


Speech analytics system for optimizing operational 
efficiencies and improving customer satisfaction; currently 
in beta supporting nearly 100 analysts 


Audio Search: Based on AT&T's WATSON technologies; 
Over one billion words processed this year. 


Customer Experience: Integrates customer experience 
surveys with call records to enable instant global and 
collaborative search among analysts 


Analytics: Supports for trend analysis and summary 
reports by call reason, regions, agent, customer 
Satisfaction metrics, etc. 
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Conclusions — 





Advances in operations over the past decade have been in 
orders of magnitude, driven by needs of applications such 
as: IP, Mobility, Web, and Sensors, and improvements in 
systems and analytic technologies. 
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Home Appliances 







Vehicles & Handheld Devices 


Pallets and Cases Computers 


Consumer Items & Sensors 
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e Nature of Analytics Changes 


e Optimize Speed vs. Accuracy Tradeoffs 
e Depth & Volatility: Rules vs. Differences 


e Data Management of near real time, Interactive, and Record & 
Playback Visualization 


e Real Time Data Quality Assessment & Improvement 
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e Source of Fundamental Automation & Organizational Change 
e Heterogeneous Event Stream Management & Query 

e Database Integration, Approximate Matching 

e Data Management for Interactive Visualization 

e Data Feed Management 


e Real time data quality 







Page 36 4/27/2010 2010 Copyright AT&T Corporation All Rights Reserved. 








Selected Research Directions 


Pervasive: 

¢ Scale 

¢ Security 

¢ Mobility 

¢ Operations 
¢ Reliability 


The Next 5 Years 


e Network Based Computing: Corporate grade “cloud” 
computing will be routinely employed in critical 
applications. Location and Presence. 


e Rich Media: Mission critical, interactive applications 
will employ multimedia and move seamlessly between 
all 3 canonical screens. 


e Networks (including Internet) of everything: Billions 
of devices, many mobile, will interact as computing, 
sensing, and communications platforms. 


e Information Leverage: Collection, Analysis, 
Visualization, & Distribution will include all forms of 
data (e.g. Relational, Semi-Structured, Text, Speech, 
Video, Image) integrated, near real time, and at huge 
scale. 


e Communities of Interest: Collaboration, Social 
Networks, Group Oriented Services. 
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